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PREFACE 


This  test  plan  will  investigate  the  Critical  Operational  Issues  and  Criteria  and  Additional 
Evaluation  Issues  and  Criteria  set  forth  by  the  Federal  Aviation  Administration.  This  plan  was 
developed  by  the  Aviation  Security  Human  Factors  Program  in  support  of  the  Trace  Detection 
Program  at  the  FAA’s  Technical  Center,  Atlantic  City  International  Airport,  New  Jersey.  Key 
FAA  personnel  supporting  this  plan  are  J.  L.  Fobes,  Ph.D.,  Aviation  Security  Human  Factors 
Program  Manager;  Susan  F.  HaUowell,  Ph.D.,  Research  Chemist;  J.  Michael  Barrientos,  Technical 
Specialist;  and  S.  Cormier,  Ph.D.,  Engineering  Research  Psychologist;  all  employed  with  the 
Aviation  Security  Research  and  Development  Division. 

Galaxy  Scientific  Corporation  (GSC)  provided  support  for  this  document  under  contract  number 
DTFA03-92-C-00035  for  the  FAA  Technical  Center.  The  co-author  of  this  document  is  Douglas 
S.  Fischer  of  GSC. 
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1.  INTRODUCTION. 


1.1  Background. 

Over  the  past  decade,  terrorist  activity  has  shifted  from  hijackings  to  the  use  of  explosives  for 
sabotaging  commercial  aircraft.  This  shift  was  highlighted  dramatically  by  the  Pan  American 
Flight  103  disaster.  Terrorists  are  now  capable  of  building  more  sophisticated  explosive  devices 
that  are  small,  easily  disguised  and  capable  of  destroying  an  aircraft,  killing  hundreds  of  lives.  As 
part  of  an  ongoing  program  to  counter  terrorist  bombings,  the  Federal  Aviation  Administration 
(FAA)  is  supporting  the  development  of  systems  to  detect  these  explosive  devices.  Yeaple  (1991) 
evaluated  the  status  of  explosives  detection  technology  and  concluded  that  ‘the  technology  of 
terrorism  has  outpaced  airport  security.”  New  detection  technologies  are  being  developed  and 
tested  but,  according  to  Lovett  (1992),  a  dog’s  nose  may  be  the  best  detector  of  explosives. 
While  most  detection  devices  are  large,  fixed-based,  and  expensive,  dogs  are  a  mobile  and 
relatively  inexpensive  explosive  detectors  that  have  been  field  tested  for  many  years  (Carr-Harris 
and  Thai,  1970;  Nolan  and  Gravitte,  1977;  Eastwood,  1990;  Francis,  1990a;  and  Francis,  1990c). 

Until  recently,  most  of  the  evidence  for  a  dog’s  keen  sense  of  smell  was  anecdotal.  Comparative 
anatomical  studies,  however,  indicate  that  dogs  have  a  highly  developed  olfactory  apparatus 
(Lovett,  1992;  Chao,  1977;  Coile,  in  preparation;  and  Syrotuck,  1972).  Mitchell  (1976)  indicates 
that  the  dogs’  olfactory  system  is  highly  selective  and  appears  to  be  sensitive  to  small  quantities  of 
relevant  target  substances.  Further,  hfrtchell  indicates  that  dogs  can  be  trained  in  olfactory 
discrimination  using  operant  conditioning  principles.  The  success  of  dogs  in  tracking  and 
detecting  contraband  items  and  explosives  suggests  that  they  are  capable  of  ignoring  distractions 
and  attending  to  signal  odorants.  Regardless,  not  much  is  known  about  the  dog’s  sensitivity  to 
odor  intensity. 

The  Office  of  the  Associate  Administrator  for  Civil  Aviation  Security  (ACS)  has  sponsored  an 
initiative  to  quantify  the  capabilities  and  limitations  of  FAA  K-9  teams  and  improve  then- 
performance.  Part  of  this  initiative  includes  conducting  an  operational  test  and  evaluation 
(OT&E)  to  determine  the  ability  of  candidate  quality  control  aids  (QCAs)  to  assess  FAA  K-9 
team  performance.  K-9  teams  are  currently  evaluated  annually  on  their  ability  to  detect  explosives 
hidden  within  the  aircraft  operations  area  (AO A)  of  U.S.  airports.  However,  there  are  no 
procedures  in  place  to  more  frequently  evaluate  the  ability  of  K-9  teams  to  meet  performance 
standards.  This  is  primarily  due  to  the  logistical  problems  associated  with  procuring  and  handling 
actual  explosives. 

The  FAA  and  Idaho  National  Engineering  Laboratory  (INEL)  vrill  develop  QCAs  that  mimic  the 
scent  of  actual  explosives.  As  simulants,  they  will  be  easier  to  handle  and  deploy  than  actual 
explosives,  pose  no  health  or  safety  risks,  and  have  a  reasonable  shelf  life.  Thus,  the  QCAs  will 
allow  the  FAA  to  determine  the  operational  readiness  of  K-9  teams  throughout  the  country  on  a 
much  more  frequent  basis  than  does  the  current  procedure. 
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1 .2  Current  Federal  Aviation  Administration  K-9  Program. 


FAA  K-9  teams  are  trained  by  the  341st  Military  Working  Dog  Training  School  (MWDTS)  at 
Lackland  AFB.  The  FAA  has  92  K-9  teams  available  in  3 1  cities  across  the  United  States.  Each 
participating  city  is  required  to  have  at  least  two  teams  that  meet  FAA  certification  standards. 

When  a  potential  threat  is  declared,  the  local  law  enforcement  uses  FAA  sponsored  K-9  teams  to 
search  the  AO  A  at  U.S.  airports  for  clandestine  ejqjlosives.  The  main  priority  of  K-9  teams  is  to 
provide  a  30-niinute  or  less  response  time  to  their  respective  airport  when  a  threat  occurs.  The 
second  priority  is  to  be  able  to  be  sent  anywhere  in  the  country  should  an  aircraft  in-flight  receive 
a  threat. 

1.2.1  Monitoring  K-9  Team  Performance.  . 

Feedback  to  the  FAA  on  K-9  team  explosives  detection  performance  is  currently  limited  to  annual 
certification.  Established  certification  procedures  test  the  K-9  teams  with  the  same  explosive 
materials  used  during  training.  These  materials,  called  actual  explosive  training  aids,  are  small 
samples  (e.g.,  1  to  2  pounds)  of  nine  different  kinds  of  explosives. 

A  potential  problem  identified  during  the  analysis  of  the  annual  certification  process  is  the  issue  of 
cross-contamination  (Cormier  et  al.,  1995).  Currently,  cross-contamination  can  occur  when 
odorants  are  combined  across  actual  explosive  training  aids.  The  problem  is  that  volatile 
components  of  actual  explosive  training  aids  may  contaminate,  dilute,  or  mask  the  odorants  of 
other  training  aids.  Age,  storage,  and  handling  potentially  contribute  to  cross-contamination 
problems.  For  instance,  the  FAA  requires  that  cities  change  their  training  aids  eveiy  four  months, 
yet  some  cities  use  training  aids  that  are  over  a  year  old.  Further,  these  training  aids  are  shipped 
and  stored  in  containers  that  are  packed  tightly  together,  possibly  allowing  the  volatile  vapors  to 
combine  across  actual  explosive  training  aids.  It  is  possible  that  the  actual  explosive  training  aids 
are  initially  contaminated,  and  that  the  dogs  are  trained  to  smell  only  the  strongest  volatile  odor. 
Therefore,  to  prohibit  the  chance  of  cross-contamination,  the  FAA  will  produce,  store,  and  ship 
the  actual  explosive  training  aids  and  QCAs  in  a  more  controlled  manner. 


1 .3  Purpose  of  The  Operational  Test  and  Evaluation. 

Through  the  OT&E,  quantified  empirical  data  will  be  obtained  to  support  the  development  of  a 
field  quality  control  procedure  to  routinely  verify  and  validate  K-9  team  performance  after  initial 
FAA  explosives  detection  training.  The  data  will  be  used  to  identify  a  relationship  between  the 
volatUe  signatures  of  the  explosive  training  aids  or  the  accompanying  particular  QCAs  detection 
performance. 
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1 .4  Scope  of  The  Operational  Test  and  Evaluation. 


The  OT&E  will  focus  on  determining  the  effectiveness  of  actual  explosive  training  aids  compared 
with  QCAs  in  meeting  the  requirements  set  forth  in  the  critical  operational  issues  and  criteria 
(COIC)  and  additional  evaluation  issues  and  criteria  (AEIC). 

Actual  explosive  training  aids  and  QCAs  will  be  tested  for  three  types  of  explosives:  double- 
based  smokeless  powder,  TNT,  and  C4.  Three  different  QCA  configurations  be  evaluated  to 
ascertain  the  optimal  configuration  for  each  explosive  type.  The  volatile  odorants  of  one  of  the 
three  explosive  types  will  be  contained  with  each  configuration  (for  a  total  of  nine  QCAs).  The 
three  QCA  configurations  include  a  cloth  patch  treated  with  a  pure  solution  (patch  1),  a  pouch 
filled  with  a  granular  composite,  and  a  non-hazardous  explosive  for  security  training  and  testing 
(NESTT)  material.  The  FAA  and  INEL  will  develop  and  provide  the  QCAs  for  the  OT&E. 

The  OT&E  also  involves  D®L  identifying  volatile  signatures  of  current  actual  explosive  training 
aids  and  candidate  QCAs.  During  the  OT&E,  INEL  will  use  a  hand-held  explosive  detection 
device,  at  the  points  where  a  dog  makes  a  detection  response,  to  determine  if  an  actual  explosive 
training  aid  or  a  QCA  exists.  INEL  will  also  use  the  device  to  determine  if  the  scent  boxes  and 
explosive  storage  areas  are  cross  contaminated.  Additionally,  INEL  will  collect  ambient  air 
samples  of  the  actual  explosive  training  aids  and  QCAs.  INEL  will  perform  qualitative  sample 
analyses  with  a  Viking  portable  mass  spectrometer  to  determine  the  volatile  signatures  of  the 
respective  explosive  training  aids  and  QCAs.  The  strength  of  the  relationship  between  the  volatile 
signatures  of  the  actual  explosive  training  aids  or  the  QCAs  and  the  detection  performance  will  be 
determined. 
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2.  CRITICAL  OPERATIONAL  ISSUES,  EVALUATION  STRATEGY,  AND  CRITERIA. 


Each  actual  explosive  training  aid  and  QCA  configuration  will  be  evaluated  against  the  COIC 
using  associated  Measures  of  Performance  (MOPs). 


2. 1  Issue  1 .  Detecting  Actual  Explosive  Training  Aids  and  Quality  Control  Aids. 

What  is  the  probability  of  detection  (Pd)  and  the  probability  of  false  alarm  (Pu)  for  dogs  detecting 
actual  explosive  training  aids  and  QCA  configurations? 

This  issue  will  be  addressed  by  recording  the  number  of  hits  (Nh)  and  the  number  of  false  alarms 
(Nfa)  for  dogs  detecting  both  actual  explosive  training  aids  and  QCA  configurations  using  a  10 
scent  box  protocol.  Appendk  A  contains  a  description  of  the  10  scent  box  protocol.  The  order  of 
explosive  training  aid  and  QCA  presentation  will  be  counterbalanced  so  that  the  dogs  will  first  be 
presented  actual  training  aids  on  half  of  the  trials  and  QCAs  first  on  the  other  half 

Measures  such  as  Pd  and  P&,  conventionally  used  in  detection  studies,  unfortunately  vary 
considerably  with  changes  in  decision  strategy.  Their  usage  can  thus  give  a  distorted  picture  of 
K-9  team’s  efficacy.  For  example,  a  very  liberal  decision  strategy  (one  requiring  little  convincing) 
can  result  in  a  very  high  Pd  but  does  so  at  the  cost  of  a  Pf*  that  is  also  very  high.  Taken  to  its 
extreme,  always  saying  an  actual  explosive  or  QCA  is  present  results  in  no  misses  but  many  false 
alarms. 

Signal  Detection  Theory  (SDT)  provides  an  approach  for  distinguishing  between  accuracy  and 
decision  strategy  in  evaluating  detection  performance.  The  principal  advantage  over  conventional 
Pd  measures  is  that  SDT  provides  a  simple  quantitative  means  of  analyzing  the  two  aspects  of 
detection  performance  -  the  effectiveness  with  which  explosive  vs.  no  explosive  discriminations 
are  made  (d')  and  the  decision  criterion  (c)  used  to  select  between  alternatives.  This  MOP  data 
(below)  will  be  used  to  derive  the  Pd,  Pf*,  sensitivity  (d')  and  c  values  for  each  dogrhandler  team. 

Criterion  1.  None.  Investigative  in  nature. 

MOP  1-1 .  The  Nh  observed  for  each  dog  handler  pair  detecting  actual  explosive 
training  aids  and  QCA  configurations. 

MOP  1-2.  The  N&  observed  for  each  dog  handler  pair  detecting  actual  explosive 
training  aids  and  QCA  configurations. 

MOP  1-3.  The  d'  observed  for  each  actual  erqrlosive  training  aid  and  QCA 
configuration. 

MOP  1-4.  The  c  for  each  actual  e}q)losive  training  aid  and  QCA  configuration. 

This  issue  will  be  addressed  by  comparing  the  Pd,  Pfe,  d',  and  c  values  observed  for  dogs  detecting 
actual  explosive  training  aids  to  those  dogs  detecting  each  QCA. 
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2,2  Issue  2.  Comparison  of  Actual  Explosive  Training  Aid  and  Quality  Control  Aid  Detection 
Performance. 


Is  there  a  difference  between  detecting  actual  explosive  training  aids  and  QCAs? 

Analysis  of  variance  (ANOVA)  and  post-hoc  comparisons  will  be  conducted  to  determine  any 
statistical  differences  in  detection  performance  between  the  actual  explosive  training  aids  and  the 
QCA  configurations. 

Criterion  2.  The  dogs’  overall  Pa  for  explosive  training  aids  is  not  statistically  different  than  that 
obtained  for  QCAs. 

MOP  2-1 .  The  Nh  observed  for  each  dog  handler  pair  detecting  actual  explosive  training 
aids  and  QCA  configurations. 

Criterion  3.  The  dogs’  overall  Pfj  for  explosive  training  aids  is  not  statistically  different  than  that 
obtained  for  QCAs. 

MOP  3-1 .  The  Nf,  observed  for  each  dog  handler  pair  detecting  actual  explosive  training 
aids  and  QCA  configurations. 

Criterion  4.  The  dogs’  overall  d'  for  explosive  training  aids  is  not  statistically  different  than  that 
for  QCAs. 

MOP  4-1.  The  d' observed  for  each  actual  explosive  training  aid  and  QCA 
configuration. 

Criterion  5.  Investigative  in  nature. 

MOP  5-1 .  The  c  for  each  actual  explosive  training  aid  and  QCA  configuration. 


2.3  Issue  3.  Volatile  Signatures. 

What  are  the  volatile  signatures  of  the  actual  explosive  training  aids  and  the  QCAs? 

During  the  OT&E,  INEL  will  perform  qualitative  sample  analyses  using  a  Viking  portable 
mass  spectrometer  for  air  samples  on  actual  explosive  training  aids  and  QCA 
configurations  that  are  detected  by  the  dogs.  These  air  samples  will  be  collected  at  the 
end  of  each  day  with  the  last  scheduled  dog. 

A  complete  description  of  the  sampling  and  analysis  plan  (including  molecular  flow  and 
sampling  parameters),  analytical  methodology,  and  sampling  requirements  will  be  provided 
in  a  separate  document  prepared  by  INEL. 
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Criterion  6.  None.  Investigative  in  nature. 


MOP  6-1.  Gas  chromatography  mass  spectral  data  of  actual  explosive  training  aids  and 
QCAs  detected. 

MOP  6-2.  Spectral  pattern  quantitative  value  of  actual  explosive  training  aids  and  QCAs 
detected. 


2.4  Issue  4.  Volatile  Signatures  and  the  Detection  of  Actual  Explosive  Training  Aids  and  Quality 
Control  Aids. 


Are  volatile  signatures  systematically  associated  with  dog  performance  in  detecting  actual 
explosive  training  aids  and  QCAs? 

This  issue  will  determine  if  dog  performance  in  detecting  actual  explosive  training  aids  and  QCA 
configurations  is  associated  with  the  identified  volatile  molecules.  ANOVAs  and  post-hoc  tests 
will  be  separately  conducted  on  Pa,  Pf»,  d'  and  c. 

Criterion  7.  None.  Investigative  in  nature. 

MOP  7-1.  Chemical  identification  of  volatile  signatures  for  actual  explosive  training  aids 
and  QCAs. 

MOP  7-2.  The  Nh  observed  for  each  dog  handler  pair  detecting  actual  explosive 
training  aids  and  QCA  configurations. 

MOP  7-3.  The  Nfi  observed  for  each  dog  handler  pair  detecting  actual  explosive 
training  aids  and  QCA  configurations. 

MOP  7-4.  The  d'  observed  for  each  actual  explosive  training  aid  and  QCA 
configurations. 

MOP  7-5.  The  c  for  each  actual  explosive  trmning  aid  and  QCA  configuration. 
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3.  ADDITIONAL  EVALUATION  ISSUES,  EVALUATION  STRATEGY,  AND  CRITERIA 


Each  QCA  will  be  evaluated  against  the  AEIC  using  the  associated  MOPs. 


3,1  Issues.  Breed  of  Dog. 

Does  the  breed  of  dog  affect  performance  in  detecting  actual  explosive  training  aids  and  QCAs? 

This  issue  will  be  addressed  by  determining  if  the  breed  of  dog  has  any  effect  on  the  detection  of 
actual  explosive  training  aids  and  QCAs.  An  ANOVA  and  post-hoc  tests  will  be  conducted  to 
determine  any  statistical  differences  in  detection  performance  across  breeds.  This  issue  will  be 
investigated  only  if  the  sample  of  dogs  consists  of  more  than  one  breed. 

Criterion  8.  None.  Investigative  in  nature. 

MOP  8-1.  The  breed  of  the  dog. 

MOP  8-2.  The  Nh  observed  for  each  dog  handler  pair  detecting  actual  explosive 
training  aids  and  QCA  configurations. 

MOP  8-3.  The  Nf*  observed  for  each  dog  handler  pair  detecting  actual  explosive 
training  aids  and  QCA  configurations. 

MOP  8-4.  The  d'  observed  for  each  actual  explosive  training  aid  and  QCA 
configurations. 

MOP  8-5.  The  c  for  each  actual  explosive  training  aid  and  QCA  configuration. 


3.2  Issue  6.  AgeofDop. 

Is  the  age  of  the  dog  associated  with  performance  in  detecting  actual  explosive  training  aids  and 
QCAs? 

This  issue  will  be  addressed  by  determining  the  relationship  between  the  ages  of  the  dogs  and  their 
performance  in  detecting  actual  explosive  training  aids  and  QCAs.  To  determine  if  a  relationship 
exists,  a  regression  analysis  will  be  carried  out. 

Criterion  9.  None.  Investigative  in  nature. 

MOP  9-1.  The  age  of  the  dog. 

MOP  9-2.  The  Nh  observed  for  each  dog  handler  pair  detecting  actual  explosive 
training  aids  and  QCA  configurations. 
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MOP  9-3,  The  Nfa  observed  for  each  dog  handler  pair  detecting  actual  explosive 
training  aids  and  QC  A  configurations. 

MOP  9-4.  The  d'  observed  for  each  actual  explosive  training  aid  and  QCA 
configurations. 

MOP  9-5.  The  c  for  each  actual  explosive  training  aid  and  QCA  configuration. 


3.3  Issue?.  Experience  of  Doe. 

Is  the  total  time  a  dog  has  been  certified  associated  with  performance  in  detecting  actual  explosive 
training  aids  and  QCAs? 

This  issue  will  be  addressed  by  determining  the  relationship  between  the  length  of  time  the  dog 
has  been  certified  and  its  performance  in  detecting  actual  explosive  training  aids  and  QCAs.  This 
issue  will  be  addressed  using  the  same  evaluation  strategy  described  in  Section  3.2. 


Criterion  10.  None.  Investigative  in  nature. 


MOP  10-1 .  Total  time  each  dog  has  been  certified  to  detect  explosives. 


MOP  10-2.  The  Nh  observed  for  each  dog  handler  pair  detecting  actual  e5q)losive 
training  aids  and  QCA  configurations. 

MOP  10-3.  The  N&  observed  for  each  dog  handler  pair  detecting  actual  explosive 
training  aids  and  QCA  configurations. 


MOP  10-4.  The  d'  observed  for  each  actual  explosive  training  aid  and  QCA 
configurations. 

MOP  10-5.  The  c  for  each  actual  explosive  training  aid  and  QCA  configuration. 


3.4  Issues.  Medical  History  of  Dog. 

Is  a  dog’s  medical  history  associated  with  its  performance  in  detecting  actual  explosive  training 
aids  and  QCAs? 

This  issue  will  be  addressed  by  determining  the  strength  of  any  relationship  between  identified 
illnesses  the  dogs  have  suffered  and  their  performance  in  detecting  actual  explosive  training  aids 
and  QCAs.  This  issue  will  be  addressed  using  Spearman  Correlations  of  qualitative  data  obtained 
from  the  handler  questionnaire.  This  issue  will  be  investigated  only  if  illnesses  have  been  reported 
in  the  sample  of  dogs. 
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Criterion  1 1 .  None.  Investigative  in  nature. 


MOP  1 1-1 .  Medical  history  of  each  dog. 

MOP  1 1-2.  The  Nh  observed  for  each  dog  handler  pair  detecting  actual  explosive 
training  aids  and  QCA  configurations. 

MOP  11-3.  The  Nfa  observed  for  each  dog  handler  pair  detecting  actual  explosive 
training  aids  and  QCA  configurations. 

MOP  11-4.  The  d'  observed  for  each  actual  explosive  training  aid  and  QCA 
configurations. 

MOP  11-5.  The  c  for  each  actual  explosive  training  aid  and  QCA  configuration. 


3.4.1  Summary  of  Issues. 

A  listing  of  the  issues  and  the  respective  MOPs  is  provided  in  Table  3-1  below. 


Table  3-1.  Issues  and  Respective  Measures  of  Performance 


Issue 

Measure  of  Performance 

1 .  Actual  Explosive  Training  Aid  and  QCA  Detection 

1-1,  1-2,  1-3,  1-4 

2.  Comparison  of  Actual  Explosive  and  QCA  Detection  Performance 

2-1,  3-1,  4-1,  5-1 

3.  Volatile  Signatures 

6-1, 6-2 

4.  Volatile  Signatures  and  Detection  Performance  Relationship 

7-1,  7-2,  7-3,  7-4,  7-5 

5.  Breed  of  Dog 

8-1,  8-2,  8-3,  8-4,  8-5 

9-1,  9-2,  9-3,  9-4,  9-5 

10-1,  10-2,  10-3,  10-4,  10-5 

1  8.  Medical  History  of  Dog 

11-1,  11-2,  11-3,  11-4,  11-5 
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4.  TEST  OVERVIEW. 


4, 1  Test  Limitations  and  Impact. 
None. 


4,2  Test  and  Evaluation  Milestones. 

Table  4-1  shows  the  milestones  for  planning  and  reporting  the  OT&E  process. 


Table  4-1 .  Test  and  Evaluation  Milestones 


Milestone 

Date 

Responsible  Organization 

Test  Concept/Design  Approval 

FAA,  AAR-510 

TEP  Submitted 

31  July  95 

Contractor 

TEP  Approved 

22  November  95 

FAA,  AAR-510 

Develop  QCAs 

30  November  95 

FAA  and  Contractor 

Test  Readiness  Review 

1  December  95 

Contractor 

Pilot  Test 

27-3 1  December  95 

FAA  and  Contractor 

QCA  OT&E 

13-30  January  96 

FAA  and  Contractor 

Preliminary  Analyses 

15  March  96 

Contractor 

Test  and  Evaluation  Report 

31  April  96 

Contractor 

Evaluation  Briefing 

9  May  96 

Contractor 
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5.  OPERATIONAL  TEST  AND  EVALUATION. 


5,1  Location. 

The  OT&E  will  be  conducted  at  the  Lackland  AFB,  home  of  the  MWDTS. 


5.2  Subjects 

Six  FAA-certified  explosives  detection  dogs  will  participate  in  the  OT&E.  Four  of  these  dogs  are 
residents  for  the  37th  Security  Police  Squadron  (SPS)  at  Lackland  AFB  and  two  work  for  the 
76th  SPS  at  nearby  Kelly  AFB,  also  in  San  Antonio,  Texas.  Dogs  will  be  selected  solely  on 
availability  and  certification  status.  The  dogs’  handlers  will  read  and  sign  an  informed  consent 
form  (see  Appendix  C)  and  complete  a  questionnaire  (see  Appendix  D)  prior  to  participating  in 
the  OT&E. 

5  3  Actual  Explosive  Training  Aids.  OCAs.  and  Blank  OCAs 

Actual  explosive  training  aids  and  QCAs  will  be  composed  of  three  types  of  explosives:  double- 
based  smokeless  powder,  TNT,  and  C4.  To  ascertain  the  optimal  configuration,  three  QCA 
configurations  will  be  evaluated  -  a  cloth  patch  treated  with  a  pure  solution  (patch  1),  a  pouch 
filled  with  a  granular  composite,  and  a  non-hazardous  explosive  for  security  training  and  testing 
(NESTT)  material. 

A  blank  QCA  is  the  same  material  used  for  the  QCA  configuration  excluding  the  explosive 
composition  (the  volatile  odorant  contained  in  the  pure  solution,  granular  composite,  and  the 
NESTT  material).  The  reason  for  testing  blank  QCA  configurations  is  to  determine  whether  or 
not  the  dogs  are  cueing  on  the  “blank”  instead  of  the  simulant. 

5.4  Test  Organization. 

Five  test  administrators  will  be  required  for  this  study  shown  in  Table  5-1.  A  description  of  the 
duties,  responsibilities,  and  schedule  for  each  test  administrator  is  included  in  i'^pendbc  E  and  F. 


Table  5-1 .  Test  Administrator  Required  for  (Quality  Control  Aid 
Operational  Test  and  Evaluation 


Personnel  Number 

Title 

Agency 

1 

Test  Director 

FAA 

2 

Contractor 

3,4 

Data  Collector 

FAA 

5 

Trial  Coordinator 

Contractor 
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5.5  Test  Organization  T raining. 


The  test  organization  will  receive  a  partial-day  training  session  on  test  procedures  and  protocol 
prior  to  the  OT&E.  The  test  administrator’s  data  collection  instructions  and  a  sample  data 
collection  form  are  provided  in  Appendix  G. 


5,6  Test  Coordination  Meeting. 

Approximately  one  month  prior  to  the  OT&E,  a  test  coordination  meeting  will  be  conducted  at 
Lackland  AFB  to  set  up  the  test  equipment  and  verify  that  the  test  administrators  are  familiar  with 
the  operational  testing  protocol.  Dogs  will  niot  be  needed  for  this  meeting;  however,  the  test 
administrators  will  practice  the  test  protocol  as  if  dogs  were  present.  Test  admimstrators  will  set 
up,  label,  and  store  all  scent  boxes  in  the  appropriate  areas.  Test  administrators  will  also  ensure 
that  the  test  site  is  accessible  and  available  for  the  scheduled  test  times  for  pilot  test  and  the 
OT&E.  If  there  is  a  conflict,  and  alternate  test  site  will  be  selected  and  prepared.  Test 
administrators  will  ensure  that  the  test  area  environmental  conditions  are  consistent  for  testing. 
For  instance,  the  test  site’s  climate  control  system  must  be  between  60  and  70  degrees  Fahrenheit. 
Wind  currents  from  fans,  vents,  or  open  windows  or  doors  must  be  identified  and  ehminated. 
Handlers  participating  in  the  pilot  test  and  OT&E  will  attend  a  meeting  on  January  5  to  discuss 
the  testing  protocol.  Test  administrators  will  read  a  standardized  in-briefing  to  the  handlers  and 
will  answer  any  questions  concerning  the  schedule  or  testing  protocol  (see  i^pendix  H  for  the  in¬ 
briefing).  Handlers  will  complete  the  informed  consent  and  questionnaire  forms  and  review  the 
testing  schedules  to  ensure  that  all  individuals  are  familiar  with  their  testing  times. 


5.7  Pilot  Study. 

Approximately  2  weeks  before  the  OT&E,  a  pilot  study  will  be  conducted  at  Lackland  AFB  to 
verify  the  test  protocol  and  data  collection  procedures.  Three  dogs  experienced  in  detecting 
smokeless  powder,  TNT,  and  C4  will  participate  in  the  pilot  study.  As  these  dogs  will  not  be 
participating  in  the  OT&E,  they  do  not  need  to  be  certified  by  the  FAA  to  detect  explosives,  but 
must  be  proficient  in  detecting  at  least  one  explosive  types. 

The  schedule  of  the  pilot  test  is  provided  in  Appendix  I.  As  the  schedule  illustrates,  3  days  are 
required  for  testing.  Dogs  will  receive  two  1-hour  trials  per  day  for  a  total  of  sk  trials.  Each  trial 
is  divided  into  two  half-hour  segments.  A  segment  consists  on  one  QCA  test  and  one  actual 
explosive  test.  Each  dog  will  be  tested  in  detecting  one  type  of  explosive  training  aid  and  three 
configurations  of  QCAs  with  the  respective  explosive  odors.  Thus,  the  nine  QCAs  will  be 
presented  to  the  dogs;  however,  each  dog  will  only  receive  the  three  QCAs  treated  with  the  odor 
of  the  explosive  training  aid  that  it  is  proficient  in  detecting.  The  pilot  test  will  use  the  same 
protocol  as  the  OT&E  (see  Section  2.7). 
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5,8  Operational  Test  Protocol. 


Before  testing  begins,  participating  handlers  will  attend  an  in-briefing,  complete  the  informed 
consent  and  questionnaire,  and  review  testing  schedules.  The  test  schedule,  provided  in  Appendix 
J,  give  the  testing  times  and  location  for  each  dog  handler  pair.  Only  test  administrators  will  have 
schedules  that  list  the  explosive  type  and  QCA  configuration  to  be  presented  to  the  K-9  teams. 

The  test  protocol  consists  of  a  1-hour  testing  trial  which  is  repeated  throughout  the  course  of  the 
OT&E.  To  minimize  the  chance  of  an  order  effect,  QCA  configuration  and  explosive  type 
presentation  will  be  counterbalanced  across  dogs.  Each  trial  is  divided  into  two  half-hour 
segments.  For  each  segment,  one  QCA  test  and  one  actual  explosive  test  will  be  conducted 
where  each  dog  will  be  presented  the  same  explosive  type  for  the  actual  explosive  training  aid  and 
QCA  configuration  for  each  respective  segment.  The  dogs  will  detect  the  actual  training  aids  first 
for  half  of  the  trials  and  the  QCAs  first  for  the  other  half 

Each  of  the  actual  explosive  training  aid  trials  will  test  the  dogs’  ability  to  detect  the  same  actual 
explosive  type  as  represented  in  the  QCA  testing  trial  (i.e..  If  pure  C4  were  the  QCA 
configuration  type  tested  for  the  first  trial,  then  the  actual  explosive  training  aid  to  be  tested 
would  be  C4  for  that  trial).  The  dogs  will  also  be  tested  on  their  ability  to  detect  each  QCA 
configuration.  Three  QCA  configurations  will  be  evaluate  for  the  three  explosive  types  for  a  total 
of  nine  configurations  which  will  require  nine  trials  per  dog. 

One  hundred  and  fifty  scent  boxes  will  be  used  for  the  OT&E.  To  reduce  the  possibility  of  cross¬ 
contamination,  each  target  scent  box  will  be  used  to  contain  only  one  type  of  explosive  training 
aid  or  one  type  of  QCA.  For  the  QCA  testing,  five  boxes  will  contain  a  target  QCA  and  five  will 
contain  a  blank  QCA  of  the  same  configuration  (see  Section  5.3  on  blank  QCAs).  For  the  actual 
explosive  training  aid  testing,  five  boxes  will  contain  explosive  aids  of  the  same  type  and  five  will 
remain  empty. 

The  test  manager  will  be  responsible  for  ensuring  that  the  dogs  are  available  for  each  testing 
session.  The  trial  coordinator  will  ensure  the  appropriate  selection,  handling  -  placement,  and 
storage  of  actual  explosive  training  aids  and  QCAs  used  for  each  testing  session  (see  Appendk  E 
for  duties).  The  trial  coordinator  will  also  ensure  the  proper  markings,  selection  and  storage  of 
the  scent  boxes.  Prior  to  the  beginning  of  each  trial,  test  administrators  will  set  up  two  testing 
areas  in  an  available  theater  at  Lackland  AFB.  Appendix  K  provides  a  diagram  of  the  test  area. 
The  two  testing  areas  are  the  lobby  and  on  the  stage  of  the  theater.  Each  testing  area  will  contain 
10  scent  boxes  staggered  with  approximately  4-foot  separations  (see  Appendix  A,  Figure  A-2). 
For  each  test  area,  five  scent  boxes  will  contain  a  target  (e.g.,  actual  explosive  training  aid  or 
QCA)  and  five  will  either  remain  empty  or  contain  a  blank  QCA.  For  both  tests,  distribution  of 
the  five  target  scent  boxes  wiU  be  randomized.  A  detailed  description  of  the  10  scent  box 
protocol  is  provided  in  Appendix  A. 

Once  the  test  areas  are  set  up,  a  handler  and  dog  pair  will  enter  the  appropriate  area.  The  handler 
will  direct  the  dog  to  the  first  scent  box.  The  dog  will  place  its  nose  in  the  hole  on  the  top  of  the 
box.  If  the  dog  detects  an  explosive,  it  will  respond  by  sitting.  The  handler,  who  does  not  know 
which  scent  boxes  contain  training  aids,  will  offer  the  dog  a  reward  for  a  correct  response.  If  the 
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dog  responded  incorrectly,  a  test  administrator  will  stop  the  handler  from  giving  the  dog  a  reward 
by  saying  “No”  before  the  handler  reaches  for  the  reward.  The  handler  will  then  direct  the  dog  to 
the  next  scent  box.  This  procedure  continues  until  all  scent  boxes  have  been  examined. 

After  completing  the  first  explosive  detection  test  segment,  the  handler  will  walk  the  dog  to  the 
second  test  area,  where  the  above  test  scenario  will  be  repeated.  Upon  completing  all  trials, 
handlers  will  be  debriefed  and  asked  for  their  opinions  of  the  testing. 

5.8.1  Scent  Box  Management. 

The  341st  MWDTS  will  construct  the  boxes  to  measure  12  inches  x  12  inches  x  6  inches  with  a  6 
inches  diameter  hole  in  the  center.  The  seams  of  the  boxes  will  be  sealed  with  masking  tape  and 
store  them  in  an  area  that  has  not  been  contaminated  by  explosives.  Boxes  will  be  marked  to 
indicate  the  explosive  type  and  QCA  configuration  that  they  will  contain  (the  marking  will  be 
placed  on  the  bottomside  of  the  box,  out  of  sight  for  the  handler).  Prior  to  constructing  the 
boxes,  personnel  will  wash  their  hands  and  put  on  nylon  gloves  provided  by  INEL.  The  trial 
coordinator  will  ensure  that  gloves  are  replaced  each  time  prior  to  handling  an  explosive  training 
aid  or  QCA.  The  341st  MWDTS  will  not  put  any  explosive  training  aids,  objects,  or  substances 
in  the  boxes.  See  Appendix  A  for  the  key  as  to  how  the  boxes  will  be  marked. 


5.8.2  Ambient  Air  Sampling  Protocol. 

INEL  will  collect  air  samples  of  the  actual  explosives  and  QCA  configurations  that  the  dogs  will 
be  attempting  to  detect.  The  air  sampling  will  collected  during  the  10  scent  box  protocol  with  the 
hand-held  explosive  detection  devices  for  every  dog’s  hit  or  miss  sit  response.  Thus,  the  device 
will  be  used  to  determine  if  explosives  or  QCA  were  present  each  time  a  dog  makes  a  hit  or  false 
alarm  response.  All  dogs  will  participate  in  the  INEL  data  collection  effort. 

INEL  will  also  collect  air  samples  each  day  with  sampling  pumps  and  absorbent  traps.  The 
samples  will  be  analyzed  with  a  mass  spectrometer  to  determine  the  volatile  signatures  of  the 
explosives  and  QCAs  available  to  be  detected  by  the  last  dog.  Analysis  will  require  approximately 
2  to  4  hours  per  air  sample,  a  requirement  imposed  by  the  equipment.  Blank  air  samples  will  be 
collected  immediately  prior  to  collecting  each  of  the  actual  explosive  aid  and  QCA  samples. 
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6.  DATA. 


6, 1  Critical  Operational  Issues  Data. 

The  values  of  Nh  and  Na  will  be  used  to  derive  the  actual  explosive  training  aid  and  QCA 
detection  performance  of  each  dog.  The  training  aid  detection  variables  are  Pd,  P&,  d',  and  c  and 
their  calculated  values  will  be  compared  across  explosive  training  aid,  QCA  configuration,  and 
volatile  signature. 

The  Nh  will  be  used  to  calculate  the  Pa  as  follows: 

Pd  =  Nh/5 

where  5  is  the  total  number  of  target  presentations. 

The  N&  will  be  used  to  calculate  the  P^  as  follows: 

Pfa=N&/5 

where  5  is  the  total  number  of  empty  scent  boxes. 

The  above  probability  will  be  used  to  determine  SDT  measures  by  converting  Pa  and  P&  to 
z-scores,  where  z-scores  are  abscissa  values  fi'om  the  standard  normal  curve. 

Pa  and  Pf*  will  be  used  to  determine  a  value  of  d'  as  follows: 
d'  =  Zf.-Zh 

where  Zh  and  Z&  are  abscissa  values  of  the  normal  curve  determined  fi'om  Pa  and 
Pfe  respectively. 

Pa  and  Pa  will  be  used  to  determine  a  value  of  c  as  follows: 
c=  0.5(Zh+Zfi,) 


Volatile  signatures,  spectral  pattern,  and  spectral  pattern  retention  time  of  the  actual  explosive 
training  aids  and  QCAs  will  be  collected  and  calculated  by  INEL.  A  three-dimensional  spectral 
analysis  will  identify  the  molecules  available  for  detection  of  each  sampled  explosive  training  aid 
and  QCA  configuration. 

6.2  Additional  Evaluation  Issues  Data. 

The  breed,  age,  experience,  and  medical  histoiy  of  the  dogs  will  be  obtained  fi'om  the  handler 
questionnaire  completed  following  QCA  testing.  (A  sample  handler  questionnaire  is  provided  in 
Appendix  D.)  Performance  data  will  be  collected  and  manipulated  as  described  in  Section  6.1. 
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6.3  Test  Data  Management. 

6.3.1  Operational  Test  Data. 

The  final  operational  test  database  will  contain  the  data  required  to  statistically  analyze  the  COICs 
and  AEICs.  Table  6-1  lists  the  data  elements  to  be  included  in  the  database. 

Table  6-1.  K-9  Quality  Control  Aid  Operational 
Test  and  Evaluation  Database 


Dog  Number 


Handler  Number 


Training  Aid 


QCA  Configuration 


Molecule  Type 


Signature  Data  Smokeless  Powder  Training  Aid 


Signature  Data  TNT  Training  Aid 


Signature  Data  C4  Training  Aid 


Signature  Data  QCAl 


Signature  Data  QCA5 


Signature  Data  QCA9 


Breed  of  Do 


Age  of  Do 


Experience  of  Do 


Medical  History  of  Do 


Nh ,  Nfa,  d’,  and  c  for  the  followin 


Smokeless  Powder  Training  Aid 


TNT  Training  Aid 


C4  Training  Aid 


_ QCAl 


_ QCA  2 


QCA  3 


QCA4 


_ QCA  5 


CA6 


QCA  7 


QCA  8 


QCA  9 
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7.  DATA  ANALYSIS. 


7. 1  Critical  Operational  Issues. 

7.1.1  Comparison  of  Actual  Explosive  Training  Aid  and  Quality  Control  Aid  Detection 
Performance. 

The  Pd,  Pfi,  d',  and  c  for  dogs  detecting  actual  explosive  training  aids  and  QCA  configurations 
will  be  the  dependent  variables.  The  experimental  design  used  to  make  these  comparisons  will  be 
a  4  training  aids  (actual,  patch,  pouch,  NESTT)  x  3  explosive  types  (smokeless  powder,  TNT, 
C4)  within  subjects  design,  as  depicted  in  Table  7-1 . 

Four  separate  ANOVAs  will  determine  any  significant  differences  between  Pa,  Pfa,  d',  and  c  across 
training  aids  and  explosive  type,  according  to  Table  7-2.  Where  significant  effects  are 
determined,  Duncan  post-hoc  comparisons  will  be  used  to  isolate  significant  differences  between 
explosive  configurations. 


Table  7-1.  Experimental  Design  Used  to  Test  Training  Aid  Detection 


Actual 

Patch 

Pouch 

NEST! 

SP 

TNT 

C4 

SP 

TNT 

C4 

SP 

TNT 

C4 

SP 

TNT 

C4 

dogl 

dogl 

dogl 

dogl 

dogl 

dogl 

dogl 

dogl 

dogl 

in«Hi 

dog2 

dog2 

dog2 

dog2 

dog2 

dog2 

dog2 

dog2 

dog2 

dog2 

dog2 

dog2| 

E5H 

insBti 

IE3j3 

RtifSl 

E3S1 

ESiul 

ItBRl 

EE!^3 

1^1^] 

|dog6 

dog6 

dog6 

dog6 

dog6 

dog6 

dog6 

dog6 

dog6 

Legend:  SP  =  Smokeless  Powder 

Actual  =  Actual  Explosive  Training  Aid 


Table  7-2.  Analysis  of  Variance  Summary  for  Dogs  Detecting  Actual  Explosive 
Training  Aids  and  Quality  Control  Aids 


Source 

SS 

df 

MS 

F 

P 

Dogs  (subjects) 

SSD 

5 

msd 

Aid 

SSa 

3 

MSA 

MSa/MSaxd 

— 

Explosive  (Exp) 

SSe 

2 

MSe 

MSe/MSexd 

Aid  X  Exp 

SSaxe 

6 

MSaxe 

MSaxe/MSaxexd 

•• 

Aid  X  Dogs 

SSaxd 

15 

MSaxd 

— 

Exp  X  Dogs 

SSexd 

10 

MSexd 

— 

— 

Aid  X  Exp  X  Dogs 

SSaxexd 

30 

MSaxexd 

— 
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7.1.2  Volatile  Signatures  and  the  Detection  of  Actual  Explosive  Training  Aids  and  Quality 
Control  Aids. 

An  experimental  design  will  be  determined  by  INEL  following  the  lab  identification  of  volatile 
molecules.  Mean  performance  scores  will  be  stratified  according  to  the  identified  molecules. 
ANOVAs  will  be  conducted  on  the  dogs’  Pd,  Pfa,  d',  and  c  with  the  identified  molecules  as  the 
dependent  variable,  based  on  the  experimental  design.  The  identified  molecules  per  training  aid 
will  be  determined  subsequent  to  air  sampling. 


7.2  Additional  Evaluation  Issues. 


7.2.1  Breed. 

The  Pd,  Pfa,  d',  and  c  will  be  the  dependent  variables  for  dogs  detecting  the  actual  explosive 
training  aids  and  QCA  configurations  compared  across  breed.  The  experimental  design  used  to 
make  these  comparisons  will  be  a  4  training  aids  (actual,  patch,  pouch,  NESTT)  x  3  breeds 
(Labrador,  German  Shepherd,  Belgian  Malenois)  within  subjects  test,  as  depicted  in  Table  7-3. 

Table  7-3.  Experimental  Design  Used  to  Test  the  Effect  of 
Breed  on  Training  Aid  Detection 


1  Actual 

Patch 

Pouch 

] 

^STl 

Lab 

GS 

BM 

Lab 

GS 

BM 

Lab 

GS 

BM 

Lab 

GS 

BM 

dogl 

dog3 

dogS 

dogl 

dog3 

dog5 

dogl 

dog3 

dog5 

dogl 

dog3 

dogS 

dog2 

dog4 

dog6 

dog2 

dog4 

dog6 

dog2 

dog4 

dog6 

dog2 

dog4 

dog6 

Legend:  Lab  =  Labrador  BM  =  Belgian  Malenois 

GS  =  German  Shepherd  Actual  =  Actual  Explosive  Training  Aid 


ANOVAs  will  be  conducted  according  to  Table  7-4.  Where  significant  effects  are  determined, 
Duncan  post-hoc  comparisons  will  be  used  to  isolate  differences. 


Table  7-4.  Analysis  of  Variance  Summary  for  Training  Aid 
Detection  Testing  at  Lackland  Air  Force  Base 


Source 

SS 

df 

MS 

F 

P 

Dogs  (subjects) 

SSD 

5 

MSd 

«« 

— 

Aid 

SSa 

3 

MSa 

MSa/MSaxd 

— 

Breed 

SSb 

2 

MSB 

MSb/MSbxd 

— 

Aid  X  Breed 

SSaxb 

6 

MSaxb 

MSaxb/MSaxbxd 

— 

Aid  X  Dogs 

SSaxd 

15 

MSaxd 

-- 

Breed  x  Dogs 

SSbxd 

10 

MSbxd 

— 

— 

Aid  X  Breed  x  Dogs 

SSaxbxd 

30 

MSaxbxd 

— 

— 
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7.2.2  Experience. 


The  regression  analysis  will  be  carried  out  as  discussed  in  Sections  3.2  and  3.3.  This  analysis  will 
indicate  the  unique  predictive  contribution  of  age  and  experience  on  the  dogs’  performance  in 
detecting  the  actual  explosive  training  aids  and  QCAs. 


8.  PRESENTATION  OF  OPERATIONAL  TEST  AND  EVALUATION  RESULTS. 

8. 1  Critical  Operational  Issues. 

Pd,  Pft,  d',  and  c  values  of  dogs  detecting  the  actual  explosive  training  aids  and  QCAs  will  be 
reported.  Significant  effects  resulting  from  the  ANOVAs  will  be  reported,  along  with  associated 
mean  and  standard  deviations  for  each  condition.  The  results  of  the  Duncan  post-hoc 
comparisons  conducted  on  any  significant  main  effects  or  interactions  will  also  be  reported. 
Volatile  molecules  and  significant  effects,  as  determined  by  the  ANOVA,  will  be  reported  along 
with  Duncan  comparisons. 

8.2  Additional  Evaluation  Issues. 

8.2.1  Breed. 

Significant  main  effects  resulting  fi'om  the  ANOVA  carried  out  on  Pa,  ?&,  d',  and  c  across 
conditions  will  be  reported,  along  with  associated  mean  and  standard  deviations  for  each 
condition.  The  results  of  the  Duncan  post-hoc  comparisons  conducted  on  any  significant  main 
effects  will  also  be  reported. 

8.2.2  Experience. 

The  regression  coefficient  (R)  and  the  coefficient  of  determination  (R^)  will  indicate  the  strength 
of  the  relationship  between  Pa,  P^,  d',  and  c,  and  age  and  experience. 
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APPENDIX  A  - 10  Scent  Box  Protocol 


A  10  scent  box  protocol  will  be  used  during  this  study.  Each  trial  consists  of  five  boxes  with  the 
odor  in  them  and  five  imcontaminated  boxes.  Figure  A-1  provides  a  diagram  of  a  scent  box, 
while  Figure  A-2  provides  a  diagram  of  how  the  boxes  will  be  arranged.  For  the  QCA  test,  a  set 
pf  five  imcontaminated  boxes  will  contain  a  blank  QCA  of  the  same  configuration.  A  blank 
QCA  is  the  same  material  used  for  the  QCA  configuration  excluding  the  explosive  composition 
(the  volatile  odorant  contained  in  the  pure  solution,  granular  composite,  and  the  NESTT 
material).  The  reason  for  testing  blank  QCA  configurations  is  to  determine  whether  or  not  the 
dogs  are  cueing  on  the  “blank”  instead  of  the  simulant.  For  the  actual  explosive  training  aid 
testing,  the  five  imcontaminated  boxes  will  remain  empty. 

To  begin  testing:  Start  with  the  dog  in  the  sit  position,  one  step  away  fi'om  the  first  box  (see 
Figure  A-3).  Have  the  handler  present  the  box  to  the  dog  ensuring  that  it  sniffs  in  or  above  the 
hole  (see  Figure  A-4).  Allow  the  dog  time  to  respond,  but  do  not  allow  the  dog  to  walk.  If  the 
dog  exhibits  a  correct  sit  response,  present  primary,  then  secondary  reward  (see  Figures  A-5  and 
A-6).  If  the  dog  exhibits  an  incorrect  response,  pull  the  dog  out  of  this  response.  Continue  to  the 
next  scent  box.  Make  sure  the  canine  sniffs  in  all  boxes  that  are  presented. 

The  bottom  of  the  scent  boxes  will  be  marked  with  black  magic  marker  to  indicate  what  type  of 
target  odor  it  contains.  Table  A-1  shows  how  the  boxes  should  be  labeled  and  the  quantity 
needed: 
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TABLE  A-1 .  Box  Labels  And  Quantity  Of  Boxes  With  Each  Label 


Box  Label 

Quantity 

PP-SP 

5 

PP-SP-B 

5 

PP-TNT 

5 

PP-TNT-B 

5 

PP-C4 

5 

PP-C4-B . 

5 

PC-SP 

5 

PC-SPtB 

5 

PC-TNT 

5 

PC-INT-B 

5 

PC-C4 

5 

PC-C4-B 

5 

N-SP 

5 

N-SP-B 

5 

N-TNT 

5 

N-TNT-B 

5 

N-C4 

5 

N-C4-B 

5 

SP 

5 

SP-B 

5 

TNT 

5 

TNT-B 

5 

C4 

5 

C4-B 

5 

12" 


FIGURE  A- 1.  Scent  Box 


A-3 


3 


5 


FIGURE  A-2.  Proposed  Scent  Box  Arrangement  to  Conduct 
the  10  Scent  Box  Protocol 
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FIGURE  A-5.  Dog  Detects  Target  Odor  and  Provides  Sit  Response 


A-7 


FIGURE  A-6.  Handler  Provides  Primary  Reward  to  Dog  For  Detecting  Target  Odor 
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APPENDIX  B  -  Signal  Detection  Theory 


The  Signal  Detection  Theory  tSDTI  Paradigm. 

Signal  Detection  Theory  (SDT)  is  a  mathematical  representation  of  signal  detection  performance 
in  deciding  whether  or  not  a  signal  is  present.  An  operational  example  of  SDT  is  an  explosives 
detection  dog  engaged  in  olfactory  tasks  to  detect  explosives.  The  explosives  hidden  in  various 
scent  boxes  represent  signals  that  the  dogs  must  detect. 

There  are  two  response  categories  that  represent  a  dog’s  detection  performance:  yes  (an 
explosive  signal  was  present),  or  no  (an  explosive  signal  was  not  present).  There  are  also  two 
signal  presentation  states,  indicating  that  the  explosive  signal  was  present  (signal)  or  absent 
(noise).  A  combination  of  dog-handler  responses  and  the  signal  state  produces  a  2  x  2  matrix 
(Figure  B-1),  generating  four  classes  of  responses,  labeled  as  hits,  misses,  false  alarms,  and 
correct  rejections  (Wickens,  1992).  A  hit  will  be  recorded  when  a  dog-handler  team  correctly 
detects  an  explosive  hidden  in  a  scent  and  a  false  alarm  will  be  recorded  when  a  team  detects’  an 
explosive  hidden  in  a  scent  box  when  none  is  present. 


State  of  Explosive  Signal 


Yes 


Dog 

Response 


No 


FIGURE  B-1.  2x2  Matrix  of  Dog  Team  Responses  and  State  of  Explosive  Signal 


Explosive  Present  Explosive  Not  Present 


Hit 

False  Alarm 

Miss 

Coirect  Rejection 
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As  indicated  by  Wickens  (1992),  the  SDT  paradigm  assumes  that  there  are  two  stages  of 
information  processing  in  all  detection  tasks;  sensory  evidence  is  aggregated  concerning  the 
presence  or  absence  of  the  signal,  and  a  decision  is  made  about  whether  this  evidence  constitutes 
a  signal.  According  to  SDT,  external  stimuli  generate  neural  activity  in  the  brain.  On  the 
average,  there  will  be  more  sensory  or  neural  evidence  in  the  brain  when  a  signal  is  present  than 
when  it  is  absent.  This  neural  evidence,  X,  referred  to  as  the  evidence  variable,  represents  the 
rate  of  firing  of  neurons  in  the  brain.  The  response  rate  for  detecting  X  increases  in  magnitude 
with  stimulus  (signal)  intensity.  Therefore,  if  there  is  enough  neural  activity,  X  exceeds  a  critical 
threshold,  X^,  and  the  dog  provides  a  sit  response  indicating  yes,  there  is  an  explosive.  If  there  is 
too  little,  the  dog  decides  no  and  continues  with  the  explosive  search.  The  dog’s  response  is  then 
interpreted  by  its  handler.  Because  the  amount  of  energy  in  the  signal  is  typically  low,  the 
average  amount  of  X  generated  by  signals  in  the  environment  is  not  much  greater  than  the 
average  generated  when  no  signals  are  present  (noise).  Furthermore,  the  quantity  of  X  varies 
continuously,  even  in  the  absence  of  a  signal,  because  of  random  variations  in  the  environment 
and  the  dog’s  level  of  neural  firing  (i.e.,  the  neural  noise  in  the  dog’s  sensory  channels  and 
brain). 

The  relationship  between  the  presence  and  absence  of  a  signal  can  be  seen  in  the  hypothetical 
noise  and  signd  plus  noise  distributions  contained  in  Figure  B-2.  The  intersection  of  the  two 
curves  represents  the  location  where  the  probability  of  a  signal  equals  the  probability  of  noise. 
The  criterion  value,  X^,  chosen  by  the  operator,  is  shown  by  the  vertical  line.  All  X  values  to  the 
right  (X  >  Xg)  will  cause  the  operator  to  respond  “yes.”  All  X  values  to  the  left  generate  “no” 
responses.  Ilie  different  shaded  areas  represent  the  occurrences  of  hits,  misses,  false  alarms,  and 
correct  rejections. 


FIGURE  B-2.  Hypothetical  Signal  Detection  Theory  Distributions  (Wickens,  1992) 
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Emcedures  to  Calculate  SDT  Probabilities. 


a.  In  SDT,  the  detection  values  are  expressed  as  probabilities. 

b.  The  probability  of  a  hit  (P|d,  miss  (P,J,  false  alarm  (Pfa),  and  correct  rejection  (?„) 
are  determined  by  dividing  the  number  of  occurrences  in  a  cell  (Figure  B-1)  by 
the  total  number  of  occurrences  in  a  column. 

c.  The  Pi,  (also  referred  to  as  the  probability  of  detection  [PJ)  will  be  calculated  by 
dividing  the  number  of  explosives  detected  (number  of  hits)  by  the  total  number 
of  hits  and  misses: 

Pm=l-Pd 

d.  The  Pfa  will  be  determined  by  the  number  of  false  alarms  divided  by  the  total 
number  of  false  alarms  and  correct  rejections: 

Pcr“  1  •  Pfa 


Sensitivity  TdT. 

Sensitivity  refers  to  the  average  amount  of  sensory  activity  generated  by  a  given  signal  as 
compared  with  the  average  amount  of  noise-generated  activity  (Coren  and  Ward,  1989). 
Sensitivity,  d',  corresponds  to  the  separation  of  the  means  of  the  signal  and  noise  distributions 
(Figure  B-2).  As  the  magnitude  of  the  signal  increases,  the  mean  of  the  signal  distribution  moves 
to  the  right.  The  proportion  of  signals  detected  (the  P^)  changes  as  the  distance  between  the 
signed  and  noise  distributions  varies.  According  to  Wickens  (1992),  if  the  separation  between  the 
distributions  is  great,  sensitivity  is  great  and  individuals  can  quite  readily  distinguish  a  signal 
plus  noise  event  from  a  noise  only  event.  Similarly,  if  the  separation  between  signal  and  noise  is 
small,  d'  measures  will  be  low. 

Table  B-1  provides  a  representative  table  of  Z  values  and  ordinate  values  of  the  probability 
distribution  related  to  hit  and  false  alarm  responses.  A  complete  table  of  the  ordinate  values  of 
the  standard  normal  distribution  will  be  used  to  calculate  d'  for  the  Test  and  Evaluation  Plan. 

The  procedures  required  to  calculate  d'  are  as  follows  (Coren  and  Ward,  1989): 

a.  Find  the  false  alarm  rate  from  the  outcome  matrix  in  the  HIT/FA  colunm  of  Table 
B-1. 

b.  Read  across  the  table  to  the  Z  column  (the  label  of  the  abscissa  of  the  graph)  and 
write  down  the  value. 

c.  Repeat  these  operations  for  the  hit  rate,  calling  the  tabled  value  Zj,. 

d.  Calculate  d'  using  the  following  equation:  d'  =  Z^-  Zj,. 
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Handler  Response  Criterion  (g) 


In  a  signal  detection  task,  handler  decision  making  may  be  described  in  terms  of  an  handler 
response  criterion.  Handlers  may  use  risky  response  strategies  by  responding  yes  more  often 
than  no.  A  risky  strategy  allows  handlers  to  decide  on  most  of  the  responses  they  receive  from 
their  K-9,  but  also  produces  many  false  alarms.  Alternatively,  handlers  may  use  conservative 
strategies,  saying  no  most  of  the  time,  making  few  false  alarms,  but  missing  many  of  the  signals. 

One  recent  parametric  measure  of  response  bias  is  c  (Ingham,  1970;  Macmillan  & 
Creelman,1990;  Snogdgrass  &  Corwin,  1988).  The  chief  diffemce  between  the  measure  &  and  its 
parametric  alternative  B  lisein  the  maimer  in  which  they  locate  the  observer’s  criterion.  Whereas 
the  bias  index  B  locates  the  observer’s  criterion  by  the  ratio  of  the  ordinates  of  the  signal-plus- 
noise  (SN)  and  noise  (N)  distributions,  £  locates  the  criterion  by  its  distance  from  the  intersection 
of  the  two  distributions  measure  in  z-score  units.  The  intersection  defines  the  point  where  bias  is 
neutral,  and  the  location  of  the  criterion  at  that  point  yields  a  £  value  of  0.  Conservative  criteria 
yield  positive  £  values,  and  liberal  criteria  produce  negative  £  values.  The  measure  £  is  computed 
as  follows: 


c  =  .5(Zfe  +  Zh) 


Table  B-1.  Representative  Z-Scores  Of  The  Normal  Curve  For 
Different  Response  Probabilities  Used  To  Calculate  D' 


HIT/FA 

Z 

HIT/FA 

Z 

.01 

2.33 

.50 

0.00 

.02 

2.05 

.55 

-0.12 

.03 

1.88 

.60 

-0.25 

.04 

1.75 

.65 

-0.38 

.05 

1.64 

.70 

-0.52 

.08 

1.40 

.75 

-0.67 

.10 

1.28 

.80 

-0.84 

.13 

1.13 

.82 

-0.92 

.15 

1.04 

.85 

-1.40 

.18 

0.92 

.88 

-1.18 

.20 

0.84 

.90 

-1.28 

.25 

0.67 

.92 

-1.40 

.30 

0.52 

.95 

-1.64 

.35 

0.38 

.96 

-1.75 

.40 

0.25 

.97 

-1.88 

.45 

0.12 

.98 

-2.05 

.50 

0.00 

.99 

-2.33 
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APPENDIX  C  -  Informed  Consent 


I, _ ,  have  received  a  briefing  by  the  FAA 

representative  as  to  the  pxirpose  of  the  FAA  study.  I  understand  the  purpose  of  the  study  and 
have  been  provided  with  the  opportunity  to  ask  questions.  The  FAA  representative  informed  me 
that  the  study  will  require  approximately  12  hours,  potentially  spread  over  several  days. 

I  understand  that  this  study  will  impose  very  little  stress  and  will  not  affect  my  K-9  handler  status 
in  any  way.  As  part  of  the  data  analysis,  my  data  will  be  combined  with  that  of  other  individuals 
and  I  will  no  longer  be  identifiable  as  a  participant.  I  have  been  informed  that  my  name  will 
remain  CONFIDENTIAL. 

I  have  been  informed  that  I  have  the  right  to  withdraw  from  the  experiment,  and  that  the  test 
manager  may  terminate  my  participation  in  the  interest  of  safety  and  the  experiment.  I  also 
certify  that  I  am  at  least  18  years  of  age. 

I  have  been  informed  that  if  additional  details  are  needed,  I  may  contact  any  of  the  test 
administrators  during  the  study,  or  Susan  F.  Hallowell,  Ph.D.,  (609)  485-4771,  or  Robert  L. 
Malone,  (609)  645-0900,  upon  completion  of  the  study. 


Signed:  _ 

Date:  _ / _ / 


Witness:  _ 

Date:  /  / 
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APPENDIX  D  -  Handler  Questionnaire 


DATE;_ _  SUBJECT  NUMBER: _  GENDER: _ 

1 .  How  long  have  you  been  a  certified  handler?  _ 

2.  What  is  the  date  of  your  last  certification?  _ 

3.  Have  you  ever  lost  your  certification?  Yes  No 

If  yes,  please  explain. 


4.  How  long  have  you  been  paired  with  the  dog  participating  in  this  study?  _ 

5.  How  long  has  this  dog  been  certified?  _ 

6.  Has  the  dog  ever  failed  certification?  Yes  No 

If  yes,  please  explain. 


7.  What  breed  is  this  dog?  _ 

8.  How  old  is  this  dog?  _ 

9.  What  sex  is  this  dog?  _ 

Is  your  dog  neutered?  Yes  No 

1 0.  Please  list  any  medical  problems  this  dog  has  experienced: 


1 1 .  Have  you  provided  the  test  administrator  with  a  copy  of 

this  dog’s  medical  history?  Yes  No 
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APPENDIX  E  -  Statement  Of  Duties 


Test  Director: 

Responsible  for  directing  and  overseeing  all  test  activities  and  personnel. 


Test  Manager: 


Federal  Aviation  Administration  (FAA) 


Ensures  that  detection  teams  are  greeted  and  that  they  complete  the  required  administration 
procedures. 

Conducts  briefs  and  debriefs. 

Manages  daily  test  activities  and  starts  and  stops  the  test  sequence  as  required. 

Plans  for  and  directs  contingency  activities. 

Liaisons  with  341st  Military  Working  Dog  Training  Squadron  administrative  personnel. 

Liaisons  with  Idaho  National  Engineering  Laboratory  administrative  personnel. 

Ensures  the  presence  of  the  proper  detection  team  in  the  required  location  at  the  required  time. 
Resolves  any  encountered  problems  with  the  Test  Director. 

Executes  all  required  logistical  activities  as  required. 

Galaxy  Scientific  Corporation  (Galaxy) 


Data  Collector 


Ensures  that  the  subject’s  number  is  recorded  on  test  forms. 

Ensures  that  data  collection  forms  are  organized  and  filed. 

Ensures  that  each  handler  completes  a  questioimaire. 

Briefs  and  debriefs  handler  regarding  test. 

Provides  guidance  and  assistance  to  handlers  on  matters  piertaining  to  conduct  of  test. 
Records  dog  performance  on  data  collection  form. 

Assists  Trial  Coordinator  in  preparing  for  test  trials. 

FAA  ' 


Trial  Coordinator 


Galaxy 


Plans  random  order  of  explosive  training  aid  and  quality  control  aid  (QCA)  presentation. 

Ensures  that  scent  boxes  are  loaded  with  proper  training  aid  or  QCA. 

Ensures  that  no  loaded  scent  box  is  used  for  multiple  scents. 

Places  scent  boxes  in  proper  random  order  prior  to  each  trial. 

Ensures  that  the  data  collection  form  states  the  correct  explosive  training  aid  and  QCA 
configurations. 

Ensures  proper  selection,  handling,  storage,  and  disposal  of  explosive  training  aids  and  QCAs. 


Galaxy 
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APPENDIX  F  -  QUALITY  CONTROL  AID  OPERATIONAL  TEST  AND  EVALUATION 
ADMINISTRATIVE  PERSONNEL  SCHEDULE 


Legend:  FAA  =  Federal  Aviation  Administration;  GSC  =  Galaxy  Scientific  Corporation; 

INEL  =  Idaho  National  Engineering  Laboratory;  MWDTS  =  Military  Working  Dog  Training  Squadron 


APPENDIX  G  -  Data  Collection  Instructions  And  Form 


On  the  data  collection  form,  record  the  date,  subject  number,  training  aid  type,  quality  control  aid 
(QCA),  and  the  dog’s  response  to  each  scent  box. 

Use  the  following  codes  when  recording  the  training  aid  type: 


SP 

Smokeless  Powder 

TNT 

TNT 

C4 

C4 

Use  the  following  codes  when  recording  thd  QCA  type: 


PP-SP 

Patch  with  Pure  Smokeless  Powder 

PP-TNT 

Patch  with  Pure  TNT 

PP-C4 

Patch  with  Pure  C4 

PC-SP 

Patch  with  Composite  Smokeless  Powder 

PC -TNT 

Patch  with  Composite  TNT 

PC-C4 

Patch  with  Composite  C4 

N-SP 

NESTT  with  Smokeless  Powder 

N-TNT 

NESTT  with  TNT 

N-C4 

NESTT  with  C4 

Use  the  following  codes  when  recording  the  dog’s  response: 

H  (Hit)  Canine  sits  and  there  is  a  concealed  ejqilosive. 

M  (Miss)  Canine  does  not  sit  and  there  is  a  concealed  explosive. 

FA  (False  Alarm)  Canine  sits  and  there  is  not  a  concealed  explosive. 

CR  (Correct  Rejection)  Canine  does  not  sit  and  there  is  not  a  concealed  explosive. 

For  example:  Training  Aids  _S£_ 


CR 

H 

CR 

■Bimi 

lEli 

FA 

CR 

CR 

QCAs  _ 

PP-SP_ 

■IK 

CR 

ilMM 

CR  mu  ■ 

iilll 

CR 

— 

FA 

NOTE:  Scent  boxes  containing  concealed  explosives  are  indicated  by  shading 
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EXAMPLE  DATA  COLLECTION  FORM 
TRAINING  AID  TEST 

SUBJECT  NUMBER: _ 

TRIAL  #1-  DATE: _ 

Training  Aid _  _ 1^111 _ _ _ | _  | 

TRIAL  #2  -  DATE:__ _ 

Training  Aid _  _ _ , _ il^gj _ 

TRIAL  #3-  DATE: _ 

Training  Aid _  _ ^ _ ; _ 

TRIAL  #4-  DATE: _ 

Training  Aid _  _ _ ; _ i _  | 

TRIAL  #5-  DATE: _ 

Training  Aid _  : _ ilMli _ ii^KI _ 

TRIAL  #6-  DATE: _ 

Training  Aid _  ' _ i _ i! _ 

TRIAL  #7-  DATE: _ 

Training  Aid _  _ |  [ _ I _ j _ 

TRIAL  #8-  DATE: _ 

Training  Aid _  _ ’/'..-I _ _ i  .  , _ 

TRIAL  #9-  DATE: _ 

Training  Aid _ _  |  _ 
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EXAMPLE  DATA  COLLECTION  FORM 
QCA  TEST 


SUBJECT  NUMBER: 

TRIAL  #1- DATE: _ 

Training  Aid _ 

TRIAL  #2  -  DATE:_ 

Training  Aid _ 

TRIAL  #3-  DATE: _ 

Training  Aid _ 

TRIAL  #4-  DATE: 


Training  Aid _ 

TRIAL  #5-  DATE 

Training  Aid _ 

TRIAL  #6- DATE: 

Training  Aid _ 

TRIAL  #7- DATE: 


Training  Aid _ 

TRIAL  #8-  DATE 


Training  Aid _ 

TRIAL  #9-  DATE 


Training  Aid 
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APPENDIX  H  -  Inbrief  for  K-9  Handlers 


Good  moming/aftemoon.  My  name  is  _  and  working  with  me  today  are 

_ _  and  _ .  We  would  like  to  welcome  you  and  thank  you  for 

your  participation.  As  a  member  of  a  K-9  explosives  detection  team,  you  are  vital  to  aviation  security 
and  your  participation  in  this  study  will  provide  information  to  help  the  FAA  develop  new  technologies 
to  support  your  job. 

This  study  involves  K-9  teams  from  Lackland  and  Kelly  Air  Force  Bases.  Our  goal  is  to  learn  how  well 
K-9  teams  can  detect  simulated  explosives.  This  information  will  be  used  to  measure  the  effectiveness 
of  the  explosive  training  aids  and  will  allow  the  FAA  to  develop  technologies  to  improve  those  aids. 
Your  task  is  to  detect  explosives  that  have  been  hidden  in  scent  boxes.  We  will  direct  you  to  two  testing 
areas.  Each  area  contains  10  scent  boxes.  Some  of  the  scent  boxes  contain  explosives,  but  you  will  not 
know  how  many  explosives  have  been  hidden  or  in  which  boxes  they  are  located. 

You  will  be  involved  in  four  activities: 

•  First,  you  will  be  given  an  informed  consent  form.  Signing  this  form  indicates  that  you 
understand  what  the  study  is  about,  and  that  you  agree  to  take  part  in  it. 

•  Second,  you  will  fill  out  a  handler  questionnaire  form.  This  form  will  provide  us  with 
information,  such  as  how  long  you  have  been  a  handler,  and  general  information  about  your 
dog.  This  form  has  a  3-digit  number  that  will  be  used  to  identify  you  instead  of  your  name. 
All  of  your  answers  today  will  use  this  number  and  your  name  will  be  kept  confidential. 

•  Third,  you  will  be  asked  to  detect  the  explosives  hidden  in  the  scent  boxes.  If  your  dog  sits 
in  response  to  an  explosive  scent  and  you  feel  it  is  correct,  please  provide  a  reward.  If  your 
dog  is  incorrect,  a  test  administrator  will  say  “next”  prior  to  you  rewarding  your  dog.  At  that 
point,  you  should  not  rewared  the  dog  and  proceed  to  the  next  scent  box.  Please  make  sure 
that  your  dog  is  presented  with  each  scent  box. 

•  Fourth,  upon  completing  the  testing,  you  will  be  asked  for  your  opinions  of  the  study  and 
how  your  job  could  be  improved. 

Your  performance  will  be  combined  with  the  performances  of  all  handlers  participating  in  the  study. 
Your  name,  questionnaire  answers,  and  explosives  detection  performance  will  remain  strictly 
confidential.  None  of  this  information  will  be  released  to  your  supervisors  and  will  not  affect  your  job  in 
any  way. 

We  have  designed  these  activities  to  be  enjoyable  and  to  be  low  stress.  It  should  take  you  approximately 
1  hour  to  finish  each  day  of  testing,  and  the  test  will  last  for  9  working  days.  If  you  feel  that  you  do  not 
want  to  participate  in  this  study,  or  you  want  to  end  your  participation,  you  are  free  to  do  so.  While  you 
are  here,  we  will  be  available  to  answer  any  questions  you  have.  If  you  any  questions  after  you  leave 
here  today,  you  may  call  Bob  Malone  of  Galaxy  Scientific  Corporation  or  Susan  Hallowell  of  the  FAA. 
Their  phone  numbers  are  on  the  consent  form. 

Before  you  begin,  do  you  have  any  questions  or  concerns?  If  there  are  no  more  questions,  you  can  begin 
filling  out  the  forms  and  then  we  will  begin  the  test. 
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APPENDIX  I  -  PILOT  TEST  SCHEDULE 


Dog  Number 

Date 

Time 

Place 

Activity 

Day  0 

0800  to  1600 

341st  MWDTS 

Brief  and  Admin. 

PI 

Day  1 

0730  to  0745 
0745  to  0800 
0800  to  0815 
0715  to  0830 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

P2 

Day  1 

0900  to  0915 
0915  to  0930 
0930  to  0945 
0945  to  1000 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

P3 

Day  1 

1030  to  1045 
1045  to  1100 
1100  to  1115 
1115  to  1130 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

P2 

Day  1 

1230  to  1245 
1245  to  1300 
1300  to  1315 
1315  to  1330 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

PI 

Day  1 

1400  to  1415 
1415  to  1430 
1430  to  1445 ' 
1445  to  1500 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
DetectioirTrial 

P3 

Day  1 

1530  to  1545 
1545  to  1600 
1600  to  1615 
1615  to  1630 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 
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APPENDIX  I  -  PILOT  TEST  SCHEDULE  (continued) 


Dog  Number  Date 


P3  Day  2 

PI  Day  2 

P2  Day  2 

PI  Day  2 

P3  Day  2 

P2  Day  2 


Time 

Place 

0730  to  0745 

Test  Area  1 

0745  to  0800 

Test  Area  2 

0800  to  0815 

Test  Area  1 

0715  to  0830 

Test  Area  2 

0900  to  0915 

Test  Area  1 

0915  to  0930 

Test  Area  2 

0930  to  0945 

Test  Area  1 

0945  to  1000 

Test  Area  2 

1030  to  1045 

Test  Area  1 

1045  to  1100 

Test  Area  2 

1100  to  1115 

Test  Area  1 

1115  to  1130 

Test  Area  2 

1230  to  1245 

Test  Area  1 

1245  to  1300 

Test  Area  2 

1300  to  1315 

Test  Area  1 

1315  to  1330 

Test  Area  2 

1400  to  1415 

Test  Area  1 

1415  to  1430 

Test  Area  2 

1430  to  1445 

Test  Area  1 

1445  to  1500 

Test  Area  2 

1530  to  1545 

Test  Area  1 

1545  to  1600 

Test  Area  2 

1600  to  1615 

Test  Area  1 

1615  to  1630 

Test  Area  2 

Activity  INEL 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

DetectionTrial 
Detection  Trial 
Detection  Trial  X 

Detection  Trial  '  X 
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APPENDIX  I  -  PILOT  TEST  SCHEDULE  (continued) 


Dog  Number 

Date 

Time 

Place 

Activity 

INEL 

P2 

Day  3 

0730  to  0745 
0745  to  0800 
0800  to  0815 
0715  to  0830 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

P3 

Day  3 

0900  to  0915 
0915  to  0930 
0930  to  0945 
0945  to  1000 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

PI 

Day  3 

1030  to  1045 
1045  to  1100 
1100  to  1115 
1115  to  1130 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

P3 

Day  3 

1230  to  1245 
1245  to  1300 
1300  to  1315 
1315  to  1330 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

P2 

Day  3 

1400  to  1415 
1415  to  1430 
1430  to  1445 
1445  to  1500 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

PI 

Day  3 

1530  to  1545 
1545  to  1600 
1600  to  1615 
1615  to  1630 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection-Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

X 

X 

Day  4 

0800  to  1600 

341st  MWDTS 

Debrief  and  Admin. 
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APPENDIX  J  -  Quality  Control  Aid  Operational  Test  And  Evaluation  Schedule 


Dog  Number 

Date 

Time 

Place 

Activity 

Day  0 

0800-1600 

341st  MWDTS 

Brief  and  Admin. 

1 

Day  1 

0730  to  0745 
0745  to  0800 
0800  to  0815 
0715  to  0830 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

2 

Day  1 

0900  to  0915 
0915  to  0930 
0930  to  0945 
0945  to  1000 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

3 

Day  1 

1030  to  1045 
1045  to  1100 
1100tolll5 
1115  to  1130 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

4 

Day  1 

1230  to  1245 
1245  to  1300 
1300  to  1315 
1315  to  1330 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

5 

Day  1 

1400  to  1415 
1415  to  1430 
1430  to  1445 
1445  to  1500 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection.Trial 

6 

Day  1 

1530  to  1545 
1545  to  1600 
1600  to  1615 
1615  to  1630 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

INEL 


X 

X 
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APPENDIX  J-  QCA  TESTING  SCHEDULE  (continued) 


Dog  Number 

Date 

Time 

Place 

Activity 

2 

Day  2 

0730  to  0745 
0745  to  0800 
0800  to  0815 
0715  to  0830 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

3 

Day  2 

0900  to  0915 
0915  to  0930 
0930  to  0945 
0945  to  1000 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

4 

Day  2 

1030  to  1045 
1045  to  1100 
1100  to  1115 
1115  to  1130 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

5 

Day  2 

1230  to  1245 
1245  to  1300 
1300  to  1315 
1315  to  1330 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

6 

Day  2 

1400  to  1415 
1415  to  1430 
1430  to  1445 
1445  to  1500 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

1 

Day  2 

1530  to  1545 
1545  to  1600 
1600  to  1615 
1615  to  1630 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

INEL 


X 

X 
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APPENDIX  J  -  QCA  TESTING  SCHEDULE  (continued) 


Dog  Number 

Date 

Time 

Place 

Activity 

3 

Day  3 

0730  to  0745 
0745  to  0800 
0800  to  0815 
0715  to  0830 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

4 

Day  3 

0900  to  0915 
0915  to  0930 
0930  to  0945 
0945  to  1000 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

5 

Day  3 

1030  to  1045 
1045  to  1100 
1100  to  1115 
1115  to  1130 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

6 

Day  3 

1230  to  1245 
1245  to  1300 
1300  to  1315 
1315  to  1330 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

1 

Day  3 

1400  to  1415 
1415  to  1430 
1430  to  1445 
1445  to  1500 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

2 

Day  3 

1530  to  1545 
1545  to  1600 
1600  to  1615 
1615  to  1630 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection.Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

J-3 


INEL 


X 

X 


APPENDIX  J  -  QCA  TESTING  SCHEDULE  (continued) 


Dog  Number 

Date 

Time 

Place 

Activity 

4 

Day  4 

0730  to  0745 
0745  to  0800 
0800  to  0815 
0715  to  0830 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

5 

Day  4 

0900  to  0915 
0915  to  0930 
0930  to  0945 
0945  to  1000 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

6 

Day  4 

1030  to  1045 
1045  to  1100 
1100  to  1115 
1115  to  1130 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

1 

Day  4 

1230  to  1245 
1245  to  1300 
1300  to  1315 
1315  to  1330 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

2 

Day  4 

1400  to  1415 
1415  to  1430 
1430  to  1445 
1445  to  1500 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

3 

Day  4 

1530  to  1545 
1545  to  1600 
1600  to  1615 
1615  to  1630 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection-Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

INEL 


X 

X 


J-4 


APPENDIX  J  -  QCA  TESTING  SCHEDULE  (continued) 


Dog  Number 

Date 

Time 

Place 

Activity 

5 

Day  5 

0730  to  0745 
0745  to  0800 
0800  to  0815 
0715  to  0830 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

6 

Day  5 

0900  to  0915 
0915  to  0930 
0930  to  0945 
0945  to  1000 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

1 

Day  5 

1030  to  1045 
1045  to  1100 
1100  to  1115 
1115  to  1130 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

2 

Day  5 

1230  to  1245 
1245  to  1300 
1300  to  1315 
1315  to  1330 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

3 

Day  5 

1400  to  1415 
1415  to  1430 
1430  to  1445 
1445  to  1500 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

4 

Day  5 

1530  to  1545 
1545  to  1600 
1600  to  1615 
1615  to  1630 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

INEL 


X 

X 


J-5 


APPENDIX  J  -  QCA  TESTING  SCHEDULE  (continued) 


Dog  Number 

Date 

Time 

Place 

Activity 

6 

Day  6 

0730  to  0745 
0745  to  0800 
0800  to  0815 
0715  to  0830 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

1 

Day  6 

0900  to  0915 
0915  to  0930 
0930  to  0945 
0945  to  1000  . 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

2 

Day  6 

1030  to  1045 
1045  to  1100 
1100  to  1115 
1115  to  1130 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

3 

Day  6 

1230  to  1245 
1245  to  1300 
1300  to  1315 
1315  to  1330 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

4 

Day  6 

1400  to  1415 
1415  to  1430 
1430  to  1445 
1445  to  1500 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

5 

Day  6 

1530  to  1545 
1545  to  1600 
1600  to  1615 
1615  to  1630 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

INEL 


X 

X 


J-6 


APPENDIX  J  -  QCA  TESTING  SCHEDULE  (continued) 


Dog  Number  Date _ Time  Place  Activity 


1 

Day  7 

0730  to  0745 
0745  to  0800 
0800  to  0815 
0715  to  0830 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

2 

Day  7 

0900  to  0915 
0915  to  0930 
0930  to  0945 
0945  to  1000 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

3 

Day  7 

1030  to  1045 
1045  to  1100 
1100  to  1115 
1115  to  1130 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

4 

Day  7 

1230  to  1245 
1245  to  1300 
1300  to  1315 
1315  to  1330 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

5 

Day  7 

1400  to  1415 
1415  to  1430 
1430  to  1445 
1445  to  1500 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

6 

Day  7 

1530  to  1545 
1545  to  1600 
1600  to  1615 
1615  to  1630 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection.Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

INEL 


X 

X 


J-7 


APPENDIX  J  -  QCA  TESTING  SCHEDULE  (continued) 


Dog  Number 

Date 

Time 

Place 

Activity 

INEL 

2 

Day  8 

0730  to  0745 
0745  to  0800 
0800  to  0815 
0715  to  0830 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

3 

Day  8 

0900  to  0915 
0915  to  0930 
0930  to  0945 
0945  to  1000 

Test  Area  1 

Test  Area  2 

Test  Area  1 
,  Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

4 

Day  8 

1030  to  1045 
1045  to  1100 
1100  to  1115 
1115  to  1130 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

5 

Day  8 

1230  to  1245 
1245  to  1300 
1300  to  1315 
1315  to  1330 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

6 

Day  8 

1400  to  1415 
1415  to  1430 
1430  to  1445 
1445  to  1500 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

1 

Day  8 

1530  to  1545 
1545  to  1600 
1600  to  1615 
1615  to  1630 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

DetectioftTrial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

X 

X 

J-8 


APPENDIX  J  -  QCA  TESTING  SCHEDULE  (continued) 


Dog  Number 
3 


4 


5 


6 


1 


2 


Date 

Time 

Place 

Activity 

INEL 

Day  9 

0730  to  0745 
0745  to  0800 
0800  to  0815 
0715  to  0830 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

Day  9 

0900  to  0915 
0915  to  0930 
0930  to  0945 
0945  to  1000 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

Day  9 

1030  to  1045 
1045  to  1100 
1100  to  1115 
1115  to  1130 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

Day  9 

1230  to  1245 
1245  to  1300 
1300  to  1315 
1315  to  1330 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

Day  9 

1400  to  1415 
1415  to  1430 
1430  to  1445 
1445  to  1500 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection  Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

Day  9 

1530  to  1545 
1545  to  1600 
1600  to  1615 
1615  to  1630 

Test  Area  1 

Test  Area  2 

Test  Area  1 

Test  Area  2 

Detection-Trial 
Detection  Trial 
Detection  Trial 
Detection  Trial 

X 

X 

Day  10 

0800  to  1600 

341st  MWDTS 

Debrief  and  Admin. 

J-9 


APPENDIX  K  -  Test  Area  For  Quality  Control  Aid  Operational 
Test  And  Evaluation 


(Available  Theater  at  Lackland  Air  Force  Base) 


TEST  AREA  2 
(Stage) 


TEST  AREA  1 
(Lobby) 


